Data processing

ABSTRACT

Data processing apparatus comprises an array of user-operable controls, the controls being adjustable by movement of a user&#39;s hand while touching a control; a detector for detecting when a user&#39;s hand is touching a control; a display screen for displaying respective screen icons associated with the controls; and a display processor, responsive to a detection that a user&#39;s hand is touching one of the controls, for altering the screen icon associated with that control.

This is a Continuation of application Ser. No. 09/898,548 filed Jul. 3,2001 now abandoned, which is a Continuation of application Ser. No.09/177,401 filed Oct. 23, 1998, which is now abandoned.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to data processing.

2. Description of the Prior Art

Data processing apparatus including display screens (e.g. PC computers)are generally controlled by external control devices such as keyboards,mice etc.

If the user wishes to concentrate on data displayed on the screen, it isdifficult to look at the control devices while the user is operatingsuch devices.

One solution is to make the control devices in a predeterminedconfiguration, such as that used for standard “QWERTY” keyboards. Thisthen allows touch typing to be learned. However, learning the layout ofa complicated control device can be time-consuming and difficult.

SUMMARY OF THE INVENTION

This invention provides data processing apparatus comprising:

an array of user-operable controls, the controls being adjustable bymovement of a user's hand while touching a control;

a detector for detecting when a user's hand is touching a control;

a display screen for displaying respective screen icons associated withthe controls; and

a display processor, responsive to a detection that a user's hand istouching one of the controls, for altering the screen icon associatedwith that control.

The invention allows the user to operate a potentially complicated arrayof controls without the need to learn the control layout by heart (whichmay be impossible in some applications) or to look at the controls whilehe is operating them. The controls are adjustable by movement once theuser is touching the controls, so the invention indicates to the user assoon as he touches a control in order that the user can check that he istouching the correct control before initiating an adjustment.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the inventionwill be apparent from the following detailed description of illustrativeembodiments which is to be read in connection with the accompanyingdrawings, in which:

FIG. 1 schematically illustrates an audio mixing console;

FIG. 2 schematically illustrates a digital signal processor forming partof the audio mixing console of FIG. 1;

FIG. 3 schematically illustrates a control computer forming part of theaudio mixing console of FIG. 1;

FIG. 4 schematically illustrates the display on a display screen formingpart of the audio mixing console of FIG. 1;

FIG. 5 schematically illustrates a fader panel forming part of the audiomixing console of FIG. 1;

FIGS. 6A and 6B schematically illustrate a channel strip;

FIG. 7 schematically illustrates a proximity and touch display;

FIGS. 8A and 8B schematically illustrate a screen pop-up display;

FIGS. 9 and 10 schematically illustrate circuitry within the fader panelof FIG. 5;

FIG. 11 schematically illustrates the format of a data word transmittedby the fader panel to the control computer;

FIG. 12 is a flow chart summarising the operation of the controlcomputer;

FIG. 13 is a flow chart illustrating the processing of a serial message;

FIG. 14 schematically illustrates a colour map; and

FIG. 15 is a flow chart illustrating processing of a touch screen event.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 schematically illustrates an audio mixing console comprising atouch-sensitive display screen 10, a control computer 20, a touch-faderpanel 30, a slave display screen 40 and a signal processor 50.

The basic operation of the audio mixing console is that the signalprocessor 50 receives audio signals, in analogue or digital form, andprocesses them according to parameters supplied by the control computer20. The user can adjust the parameters generated by the control computer20 either by touching the display screen 10 or by operating the touchpanel faders 30. Both of these modes of parameter adjustment will bedescribed in detail below.

The slave screen 40 is provided to display various metering informationsuch as audio signals levels at different points within the mixingconsole.

FIG. 2 schematically illustrates the digital signal processor 50. Thedigital signal processor 50 comprises a control processor 100 forcontrolling data and filter coefficient flow within the digital signalprocessor 50, an input/output (I/O) buffer 110 for receiving parameterinformation and filter coefficients from the control computer 20 and forreturning metering information back to the control computer 20, a randomaccess memory (RAM) 120 for storing current parameter data, aprogrammable DSP unit 130, an input analogue-to-digital converter 140for converting input analogue audio signals into digital audio signals(where required) and an output digital-to-analogue converter 150 forconverting digital audio signals into output analogue audio signals(where required).

FIG. 3 schematically illustrates the structure of the control computer20. The control computer 20 comprises a central processor 200 connectedto a communications bus 210. Also connected to the communications busare: an input buffer 220 for receiving data from the fader panel 30, arandom access memory (RAM) 230, program storage memory 240, a BIOScolour map 250, a video card 260 including a video card colour map, aninput buffer 270 for receiving data from the digital signal processor 50and an output buffer 280 for transmitting data to the digital signalprocessor 50.

FIG. 4 schematically illustrates the display on the touch-sensitivedisplay screen 10.

Running vertically on each side of the display are two groups of tenchannel strips 300, laid out in an arrangement similar to the physicallayout of a conventional (hardware) audio mixing console. Each channelstrip is identical to the others (apart from adjustments which are madeby the user to the various parameters defined thereby) and the channelstrips will be described with reference to FIGS. 6A and 6B below.

In a central part of the display 310 is provided a main fader 320,routing and equalisation controls 330 and display meters 340.

The channel strips include controls which are adjustable by the user,along with visual indications of the current state of the controls(rather like a hardware rotary potentiometer is adjustable by the user,with its current rotary position giving visual feedback of the currentstate of adjustment). This feature will be shown in more detail in FIGS.6A and 6B. Accordingly, as a parameter is adjusted by the user, thecontrol computer 20 makes corresponding changes to the displayed valueon the display screen 10, and also generates a replacement set of filteror control coefficients to control the corresponding processingoperation carried out by the signal processor 50.

The meters 340 provide simple level indications for, for example, leftand right channels output by the DSP 130. (In the case, the levelinformation is transmitted from the DSP 130, via the control processor100 and the I/O buffer 110, to the input buffer 270 of the controlcomputer.)

FIG. 5 schematically illustrates the fader panel 30.

The fader panel 30 is primarily a substantially linear array of elongatetouch-sensors. The touch-sensors will be described in more detail below,but briefly they are arranged to output three pieces of information tothe control computer:

(a) whether the sensor is touched at any position along its length;

(b) the position along the length of the fader at which it is touched;

(c) a signal indicating the proximity of a user's hand to the sensor.

Suitable sensors are described in WO 95/31817.

The fader panel comprises one such sensor 350 for each channel strip onthe display screen, plus an extra sensor corresponding to the main fadercontrol 320 on the display screen.

The current level or state of a parameter control is thus shown on thescreen. The touch-screen and fader touch-sensors can be used to adjustthat current level in either direction, but this is only a relativeadjustment form the current level. In other words, a particular fingerposition on a fader touch-sensor is not mapped to a particular gainvalue for the corresponding channel, but instead finger movements on atouch-sensor are mapped to adjustments up or down in the gain value.

So, when an adjustment is to be made via the fader panel, the usertouches the appropriate fader touch-sensor (for the particular channelor the main fader to be adjusted). The user then moves his finger up ordown that touch-sensor. Whatever linear position along the sensor theuser's finger starts at, the adjustment is made with respect to thecurrent level of the gain control represented by that fader.

FIGS. 6A and 6B taken together illustrate a channel strip.

The channel strip is a schematic illustration on the display screen of anumber of audio processing controls and devices which can be placed inthe signal processing path for each of the channels. From the top ofFIG. 6A, there is an input pre-amplifier, a variable delay control, ahigh-pass filter, two band-splitting filters, three controls relating tooutput feeds from the channel, a so-called panpot, a channel label, anda channel fader. For all of the controls shown in FIG. 6A, i.e. thosewhich process different attributes of the audio signal, the controls canbe displayed either in bold or faint colour on the display screen. Wherea control is displayed in bold colour, this indicates that the controlis “in circuit”. Where a control is displayed in faint colour (so-called“greyed out”), the control can still be adjusted but it is not currentlyin the audio circuit.

As an example of the “greying out” feature, consider the “delay” controlat the second-to-top control position in the channels strip (FIG. 6A).The delay can be set to values between, say, 0 milliseconds (mS) and1000 mS whether or not the delay processor is in the audio circuit, butthe delay period is applied to the audio signal only if the delayprocessor is in circuit.

The channel strip of FIGS. 6A and 6B also illustrates how a visualfeedback of a current control setting is given to the user. All of thecontrols except for the channel fader have an associated numerical valuegiving their current setting (e.g. 60 Hz for a filter centre frequency,0.0 dB for a gain), as well as a semicircle with a pointer schematicallyillustrating the current setting with respect to the available range ofsettings in a manner similar to the hand of a clock from a lowestpossible value (pointer horizontal and to the left) to a highestpossible value (pointer horizontal and to the right). So, for the centrefrequency of upper the band splitting filter in FIG. 6A, the pointer isa third of the way around the semicircle, indicating that the currentvalue of 60 Hz is nearer to the lower extreme than to the higherextreme. The scales used to map current settings to rotary positions onthe semicircles need not be linear, but could be logarithmic orotherwise.

FIG. 7 schematically illustrates the way in which proximity and touch isdisplayed on the display screen with regard to the faders.

When one of the sensors on the fader panel 30 is touched, thecorresponding fader display on the display screen (in this example, aparticular fader 400) is coloured in a contrasting colour to the rest ofthe screen—e.g. red. This shows that that particular fader is currentlybeing touched and so is open to adjustment.

Similarly, when the user's hand is near to one of the faders (asdetected by the proximity detector—see above), that fader is coloured inone of several shades of a further contrasting colour, for examplegetting more saturated as the user's hand gets closer to that fadertouch-sensor. Examples are shown as faders 410 in FIG. 7.

This system allows the user to track his hands across the fader panel 30without having to look down at the fader panel itself, since he can seethe proximity of his hands to different faders on the screen.Furthermore, because several degrees of proximity are available fordisplay, it is possible to work out the location of the user's hand fromthe distribution of the different colours representing different degreesof proximity.

FIGS. 8A and 8B schematically illustrate a so-called screen pop-updisplay.

FIG. 8A illustrates a part of the display screen illustrated in FIG. 4,in particular a short vertical section of three channel strips. If oneof the controls on the channel strips is touched on the screen (which isa touch-sensitive screen), the screen detects the position of the touch.This position is translated by the control computer (using a look-uptable—not shown) into the identification of the corresponding control inone of the channel strips. A pup-up display, including that control, isshown and the control can be adjusted using icons on the pop-up display.

For example if the delay control 420 in FIG. 8A is touched, acorresponding “pop-up” display appears and remains displayed until theuser selects another control for adjustment or a time delay since thepop-up was touched expires. This is illustrated in FIG. 8B.

The pop-up display includes the icon representing the control which wastouched, shown in FIG. 8B as the icon 430, but to clarify that thiscontrol is under adjustment the icon is shifted diagonally downwards andto the right by a few (e.g. 1-10) pixels. The pop-up also includes thetitle of the channel and the channel number 440, together with a fader450 allowing the value of the particular control to be adjusted.

Two modes of adjustment are available to the user. In a first mode, theuser touches the control and keeps his finger on the touch-sensitivescreen. Once the pop-up has appeared, a vertical component of movementof the user's finger from the position at which he first touched thescreen will cause a corresponding movement of the schematic fader 450and a corresponding adjustment of the attribute controlled by thatcontrol.

In a further mode of operation, the user can touch and release aparticular control without moving the finger position between touch andrelease. The pop-up then appears. The user can then touch the screenwithin the pop-up and move his finger up or down to adjust the fader450. If the user touches a non-active area of the pop-up, the pop-updisappears.

Again, adjustment is via a so-called “trim” mode, whereby the adjustmentis relative to a current setting of the control, whatever position theuser's finger starts at on the screen.

FIGS. 9 and 10 schematically illustrate circuitry within the fader panel30. In FIG. 9, a particular fader sensor 500 supplies three outputs torespective analogue-to-digital converters 510, 520, 530. These threeoutputs are: the analogue position at which the fader has been touched(if it has indeed been touched), a proximity signal indicating theproximity of a user's hand to the fader, and a touch status indicatingwhether or not the fader has been touched.

Digital equivalents of these signals are multiplexed together by amultiplexer 540, with an additional, fixed, signal indicating theidentity of the channel to which the fader 500 relates. The multiplexedoutput of the multiplexer 540 is a three byte serial data word.

All of the these data words from the various channel faders are storedthen in a previous value buffer 550 (FIG. 10). Whenever a new serialword is received, it is compared by a compare-and-control logic circuit560 with the previously buffered value. If a change is detected, thecompare-and-control logic 560 causes an output circuit to transmit thethree bytes representing the channel which has changed to the controlcomputer 20.

So, a three byte word is transmitted to the control computer 20 onlywhen the status of the fader corresponding to that channel has changed.

FIG. 11 schematically illustrates the format of a data word transmittedby the fader panel to the control computer. Each byte 570 of the threebyte data word comprises a byte header 580 and a payload 590 carryinginformation about the channel. The byte header 580 for each byteidentifies which of the three bytes in the serial word is represented bythe currently transmitted data. This enables the control computer 20 todetect when it has received all three bytes of a data word.

FIG. 12 is a flow chart summarizing the operation of the controlcomputer 20.

The control computer 20 operates a repetitive loop, which starts with acheck of the input buffer 220 (at a step 600). At a step 610, thecontents of the input buffer are examined to see whether a full threebyte serial word is present. If such a word is present, the serial wordis processed at a step 620. The processing associated with step 620 willbe described in more detail with reference to FIG. 13 below.

At a step 630, metering information is read from the signal processor 50and the meters displayed on the display screen are redrawn.

At a step 640, a detection is made as to whether the touch screen hasbeen touched or an existing touch has been removed or changed inposition. If such a touch screen event is detected, the touch screenevent is processed at a step 650. The processing associated with thestep 650 will be described in more detail below with reference to FIG.15.

Finally, if any attributes associated with signal processing operationshave changed throughout the operation of the loop, the new values aretransmitted to the digital signal processor 50.

FIG. 13 is a flow chart illustrating the processing of a serial message.

At a step 700, a detection is made as to whether the proximity or touchstatus of a channel has changed, i.e. is the channel touched where itwas not touched before or has the proximity value changed. If the answeris yes, the colour map associated with particular areas of the fadercorresponding to that channel is changed at a step 710. This processwill be described in more detail with reference to FIG. 14.

At a step 720, a detection is made as to whether a double click actionhas taken place. In other words, has the touch panel been touched,released, touched and released within a predetermined period. If such anevent is detected, a channel cut control is toggled at a step 730 andthe process ends. The channel cut control switches on or off the outputof that channel. By toggling the control, if the control is currentlyoff it toggles on, and vice versa.

If a double click event is not detected, a detection is made at a step735 as to whether the panel is currently touched. If the answer is yes,a further detection is made 740 as to whether the touch is a new touch.This detection is made by examining a stored touch attribute from aprevious operation of this flow chart.

If this is a new touch, a so-called trim mode is initiated at a step750. This involves storing the position along the fader at which the newtouch has been made and mapping it to the current value of the gainparameter controlled by that fader. Thus, when (in subsequent operationsof this flow chart) the user's hand might be moved up or down the fader,adjustment is made from the current gain attribute controlled by thefader. If this is not a new touch, then at a step 760 an adjustmentmight have to be made to the gain attribute controlled by the fader, ifthe user's finger has moved up or down the fader since the lastoperation of the flow chart.

Finally, the stored previous proximity touch status and level attributesare set to those detected during the current operation of the flow chartat a step 770.

FIG. 14 schematically illustrates a colour map.

The colour map provides a mapping between so-called logical colours(indexed from 0 to 255) and values of red, green and blue for actualdisplay on the screen. So, for example, the logical colour 1 is mappedto 60R, 60G, 60B for display.

The R,G and B values are each adjustable between 0 and 255 (i.e. 8 bits)so the colour map defines a subset of 256 of the 16.7 millioncombinations of R, G and B values.

The control computer maintains two copies of the colour map. A firstcopy, the so-called “BIOS” copy, is alterable by the control computerunder program control. Alterations can then be copied across into thevideo card colour map which is actually used to map logical colours ontodisplay parameters for the display screen.

In the present embodiment, areas of the screen such as each of thechannel faders are assigned a different logical colour, even though theR, G and B values specified by those logical colours may all beinitially the same. When the display colour of an area is to be changedrapidly, for example when the touch or proximity status of a faderchanges, then instead of redrawing the area using a standard but (inthis context) relatively slow Microsoft Windows redraw command, a simplechange is made to the colour map entry for the logical colour used forthat particular area of the screen. This has almost instant effect onthe actual displayed colour.

As described above, the change is made first to the BIOS colour map andthen the change is propagated (using a standard command) to the videocard colour map.

FIG. 15 illustrates the processing relating to step 650 of FIG. 12,namely the processing of a touch screen event.

At a step 800, a check is made as to whether the screen is currently orpreviously (i.e. at the last operation of the flowchart) touched. If theanswer is yes, then processing proceeds to step 830. If the answer isno, then a check is made at a step 810 as to whether a time delay hasexpired since the screen was last touched. If not, the process ends. Ifso, then any open pop-ups are closed at a step 820 and the process ends.

At step 830 a check is made as to whether the current touch represents anew adjustment. If so, processing proceeds to steps 840 and 850 whereany existing pop-ups are closed. At a step 860 a new pop-up for the newadjustment is opened, and at a step 870 a trim operation is initiated bymapping the current setting of the selected control to the currentfinger position, so that adjustments are made in a relative, rather thanan absolute, manner as described above. The process then ends.

If this is an existing adjustment, i.e. if the finger has not left thescreen since the trim mode was set up (on a previous operation of theflow chart) then at a step 880 the current value of the control isaltered (if the finger has moved) and the corresponding display withinthe pop-up is altered at a step 890.

In further embodiments of the invention, a detection (not shown) can bemade of the average proximity value over those sensors detecting theproximity of a user's hand. The sensitivity of the proximity measurementcan be adjusted as a result of this detection. For example, if theaverage value is that of a very weak detection (suggesting that theuser's hand is far away) then the sensitivity can be increased.

Although illustrative embodiments of the invention have been describedin detail herein with reference to the accompanying drawings, it is tobe understood that the invention is not limited to those preciseembodiments, and that various changes and modifications can be effectedtherein by one skilled in the art without departing from the scope andspirit of the invention as defined by the appended claims.

1. A data processing apparatus comprising: a touch-sensitive displayscreen for displaying a set of screen icons associated respectively withan array of user-operable controls, said array of user-operable controlsbeing adjustable by movement of a user's hand on the touch-sensitivedisplay screen while touching one of said controls, wherein a particularposition of a user's hand on one of said controls adjusts a gain valuerelative to current levels and irrespective of the start position of theuser's hand on the touch-sensitive display screen; a detector fordetecting when the user's hand is touching one of said controls; and adisplay processor, responsive to a detection, such that when said user'shand touches one of said controls, said screen icon associated with saidcontrol is enlarged and highlighted on said touch-sensitive displayscreen, thereby facilitating the user's operation of said array ofcontrols, wherein when a pop-up icon, representing a control that wastouched, is shifted diagonally downwards and to the right, by between 1and 10 pixels, on the display screen, thereby indicating that thecontrol is under adjustment, and wherein particular areas are colored indifferent shades according to degrees of proximity of the user's hand tothese particular areas.
 2. The apparatus according to claim 1, whereinsaid apparatus is an audio processing apparatus and said array ofcontrols relates at least in part to a control of audio processingparameters.
 3. The apparatus according to claim 2, in which said screenicon relating to a control displays, at least an audio processingparameter associated with said control.
 4. The apparatus according toclaim 1, in which said display processor is operable to change a displaycolor of at least a part of said icon associated with said one of saidcontrols.